Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util: expose promise events #21857

Closed

Conversation

devsnek
Copy link
Member

@devsnek devsnek commented Jul 17, 2018

Refs nodejs/promises-debugging#8
Closes nodejs/promises-debugging#8

exposes env promise hooks to userland

/cc @nodejs/promises-debugging @nodejs/async_hooks @BridgeAR @ljharb

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@devsnek devsnek added semver-major PRs that contain breaking changes and should be released in the next major version. async_wrap promises Issues and PRs related to ECMAScript promises. async_hooks Issues and PRs related to the async hooks subsystem. labels Jul 17, 2018
@nodejs-github-bot nodejs-github-bot added the lib / src Issues and PRs related to general changes in the lib or src directory. label Jul 17, 2018
@devsnek devsnek added wip Issues and PRs that are still a work in progress. and removed lib / src Issues and PRs related to general changes in the lib or src directory. labels Jul 17, 2018

Local<Function> fn = env->async_hooks_promise_event_function();

v8::HandleScope handle_scope(env->isolate());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would technically need to be swapped with the line above (it’s only working because we’re doing an evil hack and look into V8’s object representations to load persistent references faster)

src/node.h Outdated
@@ -612,9 +612,20 @@ NODE_EXTERN void AtExit(void (*cb)(void* arg), void* arg = 0);
*/
NODE_EXTERN void AtExit(Environment* env, void (*cb)(void* arg), void* arg = 0);

typedef void (*promise_hook_func) (v8::PromiseHookType type,
NODE_EXTERN typedef enum {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think it really makes sense to add NODE_EXTERN to a type definition; also, the typedef is redundant, just using enum PromiseHookType { … }; should work the same way

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't compile without the typedef

@AndreasMadsen AndreasMadsen self-requested a review July 17, 2018 19:25
Copy link
Member

@benjamingr benjamingr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is backwards incompatible and will break code for a worse off default.

This also goes against the ongoing work we've been doing at @nodejs/promises-debugging and the work.

I am strictly -1 on this change before the survey we've been working on with the foundation is sent out and we've gathered sufficient data.

@benjamingr
Copy link
Member

This allows the default behaviour in node to follow the intention of js

This is also not true and was not the intention - browsers error on unhandled rejections too. One of the things we've been getting since adding the unhandledRejection hook is constant positive feedback for logging as well as negative one for not crashing.

@BridgeAR
Copy link
Member

It would have been great to discuss things like that in the promises working group. Since this is completely backwards incompatible I am strongly -1. It is not an option to remove the regular way how people deal with this.

AFAIK it is also not possible to mirror the current implementation in userland even with the events exposed.

I believe it is best to keep these things in core so everyone takes advantage of it by default. Opt-out instead of opt-in.

@jasnell jasnell requested a review from BridgeAR July 17, 2018 19:49
Copy link
Member

@BridgeAR BridgeAR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1 due to the mentioned reasons.

@benjamingr
Copy link
Member

By hte way: By the same logic we should deprecate uncaughtException, stop exiting or logging on errors and expose an async_hook for exiting.

@devsnek
Copy link
Member Author

devsnek commented Jul 17, 2018

i can add the promiseResolve back as deprecated if you want? i assumed it wasn't needed because async_hooks is experimental still...

as to the argument about how promises are supposed to work: they are explicitly intended to be able to silently fail. what browser consoles do is completely irrelevant because that's a debug facility. (we could probably do the same thing with our inspector, which would be also fine)

@devsnek
Copy link
Member Author

devsnek commented Jul 17, 2018

@BridgeAR

AFAIK it is also not possible to mirror the current implementation in userland even with the events exposed.

i tried to design this with that in mind (rejectWithNoHandler includes the rejection reason, etc) and using process.nextTick will make it identical (except for storing asyncId instead of promise)

@ljharb
Copy link
Member

ljharb commented Jul 17, 2018

@benjamingr browsers provide a console warning on unhandled rejections, which they then retroactively erase if it becomes handled - node doesn't have that capability.

@benjamingr
Copy link
Member

they are explicitly intended to be able to silently fail.

I do not believe this is the case nor does the spec make any such requirement. That said: Under all proposed implementations promises are able to silently fail - just like you can ignore synchronous errors:

process.on('uncaughtException', e => { /* console.log('https://i.imgur.com/c4jt321.png') */ });

You can ignore unhandled rejections:

process.on('uncaughtException', e => { /* console.log('https://i.imgur.com/c4jt321.png') */ });

Which will continue to work forever under all proposed changes. The question is about what the default behaviour should be.

We have a consensus at this point in Node.js that ignoring uncaught errors should not be the default behaviour.

what browser consoles do is completely irrelevant because that's a debug facility. this is all for debugging

Again, the same argument could be made for Node.js not exiting on uncaught errors. Since a significant part of our user-base use async/await it means that in practice they don't distinguish between unhandled rejections and uncaught errors in terms of how they handle errors anyway.

@benjamingr
Copy link
Member

@ljharb

@benjamingr browsers provide a console warning on unhandled rejections, which they then retroactively erase if it becomes handled - node doesn't have that capability.

The browser error handling model is completely different though - Node.js crashes on uncaught errors (by default) whereas browsers "chug on". Both behaviours have been this way for well over 8 years.

I was just pointing out browsers consider unhandled rejections as an error and log based on it. Node.js provides the "erasing" by providing the "rejectionHandled" hook which I haven't actually seen ever used but it's possible and was added in the same PR.

@devsnek
Copy link
Member Author

devsnek commented Jul 17, 2018

I do not believe this is the case nor does the spec make any such requirement.

the spec says An implementation of HostPromiseRejectionTracker must complete normally in all cases.. i would argue that killing the process is not a normal completion, but i don't really want to get into those semantics.

aside from that, unhandledRejection (with the emit on gc proposal) can't guarantee that it will be fired for the things it should be unless someone is using global.gc, which kind of undermines the debugging potential of it. By not making these (arguably confusing for that reason) apis, we can let people find the best behaviour for what they're trying to do, similar to how people often use tools like winston even though node has console.

these changes also mean that in the common case, nothing is slowed down by promise state tracking.

@benjamingr
Copy link
Member

@devsnek I encourage you to read all the discussion above as well as the nodejs/promises repo discussions about this issue.

The gist is:

  • We can't really know when a promise rejection will never be handled (halting problem)
  • We can warn when a promise is handled asynchronously (and not within a microtask) since that has been very rare in practice. We've added this and got tremendous positive community feedback.
  • GC is the only tool we have for detecting most true positives but no false positives at all. Note this has false negatives like a unhandled rejection the app keeps a global reference for. Also note this cannot be implemented in user-land since it relies on core GC behaviour.
  • Since GC isn't perfect - when a promise isn't handled synchronously there is a high true positive chance - so it makes sense to still log these.
  • We let users choose the behaviour they want in all cases. This is only about the default behaviour.

In a gist the correct behaviour we settled on was:

  • If you can prove (GC) an exception is unhandled do the same thing we do for uncaught exceptions - exit the process.
  • If you think with a high probability that a rejection is unhandled - show that as a warning since that's what most users find beneficial (we have a recent Netflix survey that also shows this). This is easy to opt out of.

It's not perfect - but sadly I think it's the best we might be able to do. You are encouraged to read the background discussions and participate in the promises-debugging team which still has an open "call for members". We'd love for more people to get involved.

@benjamingr
Copy link
Member

benjamingr commented Jul 17, 2018

the spec says An implementation of HostPromiseRejectionTracker must complete normally in all cases.. i would argue that killing the process is not a normal completion, but i don't really want to get into those semantics.

That's not HostPromiseRejectionTracker (which just tracks any rejections) - we are strictly talking about code that happens after that when the promise gets GCd. At this point control has been yielded to the platform and only platform code is running. An implementation may do as it pleases. This is entirely Node.js's domain and not ECMAScript's.

@devsnek
Copy link
Member Author

devsnek commented Jul 17, 2018

Also note this cannot be implemented in user-land since it relies on core GC behaviour.

the destroy hook lets us do that right?

in any case, wasn't the whole point here to catch all the bad things happening? if we don't catch them we aren't solving the problem (with regard to gc) and having that as a default is imo confusing and misleading (not to mention that having all these promise state trackers by default is annoying)

@devsnek devsnek force-pushed the feature/async_hooks-promise-event branch 2 times, most recently from f83beed to ff94f4c Compare July 17, 2018 22:39
@devsnek devsnek removed the semver-major PRs that contain breaking changes and should be released in the next major version. label Jul 17, 2018
@devsnek
Copy link
Member Author

devsnek commented Jul 17, 2018

@benjamingr @BridgeAR unhandledRejection event remains. This PR still provides a solution to exposing the events requested in nodejs/promises-debugging#8 (and other fun stuff)

i still strongly recommend against node defining what an unhandled rejection is in any case but that's a conversation for another issue/pr

@devsnek devsnek removed the wip Issues and PRs that are still a work in progress. label Jul 17, 2018
doc/api/util.md Outdated
* `enable` {Function} Enables the hook
* `disable` {Function} Disables the hook

> Stability: 1 - Experimental
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This should likely be moved above the arguments list

@devsnek devsnek force-pushed the feature/async_hooks-promise-event branch from 7ff1627 to 3dec7bb Compare August 9, 2018 23:25
@devsnek
Copy link
Member Author

devsnek commented Aug 9, 2018

@mcollina
Copy link
Member

mcollina commented Aug 10, 2018

I've signed off both PR (this and #22218), but we should really reconcile them before landing either one. See @jasnell comment #22218 (comment).

@Trott
Copy link
Member

Trott commented Aug 13, 2018

@BridgeAR Same question as in the other PR: Can you explain what you'd like the TSC to decide or do with this at the meeting? (Or should we remove it from the agenda?)

EDIT: Are you asking the TSC to decide whether we should go with this approach or #22218? Is it unlikely that you and @devsnek or others will come to an agreement on that?

@Trott
Copy link
Member

Trott commented Aug 25, 2018

If I've understood the conversation above, there are currently 3 objecting Collaborators on this PR: @benjamingr, @BridgeAR, and @AndreasMadsen. If I've misunderstood and any of you don't object to this at this time, please comment! Thanks!

@Trott Trott added the blocked PRs that are blocked by other issues or PRs. label Aug 25, 2018
@Trott
Copy link
Member

Trott commented Aug 25, 2018

Adding blocked here and in #22218 pending decision about which approach to go with.

@devsnek
Copy link
Member Author

devsnek commented Aug 29, 2018

@nodejs/tsc PTAL.

@benjamingr your review doesn't show up anymore but i assume it is outdated.

@benjamingr benjamingr dismissed their stale review August 29, 2018 13:10

Like Gus said, my review is outdated.Both this and the other PR LGTM.

@fhinkel
Copy link
Member

fhinkel commented Aug 29, 2018

Thanks everybody for putting so much work into this! Would somebody be willing to summarize #22218 and #21857 so it's easier to understand the pros and cons of the two approaches?

@Trott Trott removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Aug 29, 2018
@devsnek devsnek requested a review from a team September 5, 2018 01:40
@devsnek devsnek added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Sep 5, 2018
@Trott Trott removed the request for review from a team September 8, 2018 01:48
@addaleax addaleax removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Sep 17, 2018
@addaleax
Copy link
Member

@devsnek This would at least need a rebase?

@devsnek
Copy link
Member Author

devsnek commented Sep 17, 2018

that's more than I'm willing to do at this point.

@devsnek devsnek closed this Sep 17, 2018
@devsnek devsnek deleted the feature/async_hooks-promise-event branch September 17, 2018 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked PRs that are blocked by other issues or PRs. promises Issues and PRs related to ECMAScript promises. semver-minor PRs that contain new features and should be released in the next minor version. util Issues and PRs related to the built-in util module.
Projects
None yet
Development

Successfully merging this pull request may close these issues.